Computing tips for awesome research and an easy life
Robert Turner, University of Sheffield RSE Team September, 2021
Acknowledgements
Contains elements from Reproducible Research Data and Project Management in R, by Anna Krystalli and from Methods in Research Software Engineering by David Wilby.
About me
Mix of software engineering and research experience.
RSE Team
13 RSEs, 35 projects / year worth ~£11m total
This presentation is flawed
Focusses on what to do, not how to do it.
In this session…
Discussion
What are the characteristics of well engineered research software?
Link to interactive doc
“Good” research software
- Version control
- Automated tests
- Controlled execution environment
- Documentation
- Parameterised
Data Management
Data Management Plan
- Start early. Make an RDM plan before collecting data.
- Anticipate data products as part of your thesis outputs.
- Think about what technologies to use.
Own your data
Act as though every short term study will become a long term one @tomjwebb. Needs to be reproducible in 3, 20, 100 yrs
— Oceans Initiative (@oceansresearch) January 16, 2015
Take initiative & responsibility. Think long term.
Spreadsheets?
@tomjwebb stay away from excel at all costs?
— Timothée Poisot (@tpoi) January 16, 2015
Do you agree?
Excel
THRILLED by this announcement by the Human Gene Nomenclature Committee. pic.twitter.com/BqLIOMm69d
— Janna Hutz (@jannahutz) August 4, 2020
But good for data viewing / entry, sometimes, perhaps…
Databases
@tomjwebb Entering via a database management system (e.g., Access, Filemaker) can make entry easier & help prevent data entry errors @tpoi
— Ethan White (@ethanwhite) January 16, 2015
@ethanwhite +1 Enforcing data types, options from selection etc, just some useful things a DB gives you, if you turn them on @tomjwebb @tpoi
— Gavin Simpson (@ucfagls) January 16, 2015
@tomjwebb it also prevents a lot of different bad practices. It is possible to do some of this in Excel. @tpoi
— Ethan White (@ethanwhite) January 16, 2015
Have a look at the Data Carpentry SQL for Ecology lesson
Data formats
.csv: comma separated values..tsv: tab separated values..txt: no formatting specified.
@tomjwebb It has to be interoperability/openness - can I read your data with whatever I use, without having to convert it?
— Paul Swaddle (@paul_swaddle) January 16, 2015
more unusual formats will need instructions on use.
Ensure data is machine readable
bad
bad
good
ok
- could help data entry
.csvor.tsvcopy would need to be saved.